Photo sensor array using controlled motion

ABSTRACT

A movable sensor including a plurality of photo pixel sites arranged in an array comprising a photo sensor and a neutral density filter filtering the photo sensor. Each of the neutral density filters have a density value that are graduated over a range of densities. The sensor is linearly movable across an image. Each point in the image is exposed to at least one pixel site with the graduated density values and each of the photo pixel sites of the array is exposed to a same light input during a time span of exposure, such that the image is captured at a defined range of exposure values and can be combined into a single high dynamic range image.

REFERENCE TO RELATED APPLICATIONS

This is a divisional patent application of co-pending application Ser.No. 12/251,016, filed Oct. 14, 2008, entitled “PHOTO SENSOR ARRAY USINGCONTROLLED MOTION”. The aforementioned application is herebyincorporated herein by reference.

BACKGROUND OF THE INVENTION

The present invention relates to photo sensor arrays, and morespecifically to photo sensor arrays using controlled motion andgraduated filters to capture high dynamic range images.

Current image capture devices used in consumer and professional digitalphotography rely on sensors that have a relatively low dynamic range,currently ranging up to about 11 EV. While this matches or exceeds thedynamic range of film based technologies, it still falls far short ofthe dynamic range of the human eye. High dynamic range images allow forthe capture of images that more closely match the human perception of ascene, as well as allowing for non-destructing photo image manipulation.

Two existing techniques exist for capturing high dynamic images. First,a dedicated high dynamic range photo sensor can be used. These sensorsgenerally trade resolution for dynamic range. Software exists formerging multiple low resolution images into a high dynamic range image,for example Photoshop® CS3 software, Photomatix™ software, Picturenaut™software, and HDR Shop™ software. Second, is to completely generate theimage digitally.

SUMMARY OF THE INVENTION

A movable sensor including a plurality of photo pixel sites arranged inan array comprising a photo sensor and a neutral density filterfiltering the photo sensor. Each of the neutral density filters have adensity value that are graduated over a range of densities. The sensoris linearly movable across an image. Each point in the image is exposedto at least one pixel site with the graduated density values and each ofthe photo pixel sites of the array is exposed to a same light inputduring a time span of exposure, such that the image is captured at adefined range of exposure values and can be combined into a single highdynamic range image.

A high dynamic range image can be captured using the present inventionby aligning an imaging photo sensor comprising a plurality of photopixel sites arranged in an array comprising a photo sensor and a neutraldensity filter filtering the photo sensor. Each neutral density filtershaving a density value, the density values being graduated over a rangeof densities. The photo sensor is then moved linearly across the image,such that each point in the image is exposed to at least one pixel sitehaving each of the graduated density values and each of the photo pixelsites of the array is exposed to the same light input during a time spanof exposure. Then the image is captured and the captured multipleexposure pixel data taken at multiple exposures is combined into asingle high dynamic range image using hardware or software.

In an alternate embodiment of the present invention, the photo sensor isstationary and the neutral density filter is linearly movable adjacentto and relative to the photo sensor.

A high dynamic range image can be captured using the present inventionof the alternate embodiment by aligning an imaging photo sensorcomprising a plurality of photo pixel sites arranged in an arraycomprising a photo sensor and a neutral density filter filtering thephoto sensor. Each neutral density filter having a density value thedensity values being graduated over a range of densities. The neutraldensity filter is then moved adjacent to the photo sensor and linearlyrelative thereto across the image, such that each point in the image isexposed to at least one pixel site having each of the graduated densityvalues and each of the photo pixel sites of the array is exposed to thesame light input during a time span of exposure. Then the image iscaptured and the captured multiple exposure pixel data taken at multipleexposures is combined into a single high dynamic range image usinghardware or software.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIGS. 1-4 show moving the photo sensor of an embodiment of the presentinvention with an image projected on the sensor in four differentpositions of the sensor versus the image.

FIGS. 5-7 show moving the filter of an alternate embodiment of thepresent invention with an image projected on a photo sensor in threedifferent positions of the filter versus the image.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 1-4 show an embodiment of the present invention using a movingphoto sensor 4. A photo sensor 4 has an array in which each of thecolumns 4 a linearly records red R, green G, and blue B information.Neutral density filters are added to each of the columns 4 a across arange of neutral density values to change the exposure values forindividual pixel sites without changing the color values. In otherwords, the neutral density filters make each site less sensitivity tolight without adding any color casts. For example, each column 4 a ofthe photo sensor has a pixel site for red R, green G, and blue B, withdark filters denoted as dark red DR, dark green DG, and dark blue DB,and with very dark filters denoted as very dark red VR, very dark greenVG, and very dark blue VB.

A single column 4 a of pixel sites of all three color values withfilters on the photo sensor array are recorded across a range ofexposure values. The pixel sites with the lowest neutral density filtervalues will record the darkest color values or the shadows, for examplered R, green G, and blue B, while the pixel sites with the highestneutral density values will record the brightest color values or thehighlights, (i.e. very dark red VR, very dark green VG, very dark blueVB). A wide range of dynamic values can be obtained with additionalneutral density arrays and other color patterns could be selected. Thenumber of rows or columns on the photo sensor could also vary based onthe application. It is understood that the photo sensor array orarrangement of pixel sites shown in the figures is an unrealisticallysimple array and is used for explanation purposes only. It is understoodthat a Bayer filter map arrangement may be used as well as other currentphoto sensor arrangements.

The photo sensor 4 is movable linearly across an image, for example bybeing mounted on a linear track, allowing movement in the direction ofthe neutral density striping, as shown by the arrows in FIGS. 1-4. Thephoto pixel sites have neutral density filters or graduated filters overthem in a range of neutral density values, so that each pixel is exposeda range of exposures. This linear movement allows the photo sensor 4 tobe moved such that each column 4 a of the photo sensor array is exposedto the same light input during the time span of the exposure. By varyingthe photo sensor 4 position relative to the image 2 of a smiley facerapidly during exposure, a set of images across a defined range ofexposure values can be captured and stored.

For example, in the photo sensor 4 array shown in FIGS. 1-4, the photosensor 4 would be moved across the image 2. The image 2 is captured whenthe sensor 4 is in nine distinct positions, in which the point 12 in theimage 2 would be read by photo sensor 4 as green G as shown in FIG. 1,as red R as shown in FIG. 2, as very dark blue DB as shown in FIG. 3, asvery dark green VG, as very dark red VR, as dark blue DB, as dark greenDG, as dark red DR, and as blue B as shown in FIG. 4 in turn, bylinearly moving the photo sensor 4. Each position of the photo sensor 4varies from the previous by the width of a pixel site or photo sensorsite or natural multiple thereof, so that the same point of the image 12is captured by all color channels and multiple exposure ranges.

After the multiple color and exposure values are obtained, software orhardware to take the various input values and combine them into a singlehigh dynamic range image is used. The conversion to a high dynamic rangeimage utilizes the multiple exposure and color channel values obtainedby the photo sensor during the linear travel. No interpolation or“demosaicing” is required, nor do multiple exposure have to be alignedand combined to obtain high dynamic range values. The input values areall available from the input device and can be combined withoutintroducing additional estimation errors.

Alternatively, the neutral density filters, or the gradient filters maybe combined into the color photo pixel sites of varying densities,instead of being isolated from the colored photo pixel sites.

In a second embodiment of the present invention, a movable neutraldensity filter 8 with a range of neutral density values 8 a, 8 b, 8 c isaligned with a photo sensor 4 to change the exposure values forindividual pixel sites without changing the color values is moved acrossan image 2. In other words, the neutral density filter 8 make sites onthe photo sensor 6 less sensitive to light without adding any colorcasts. The photo sensor 6 has an array in which each of the columns 6 alinearly records red R, green G, and blue B information. The stripes 8 cof the neutral density filter with the lowest density values will recordthe darkest color values or the shadows and the stripes 8 a of theneutral density filter with the highest density values will record thebrightest color values or highlights.

This neutral density filter, movable linearly across the image, forexample by being mounted on a linear track, allows movement in thedirection of the neutral density striping, as shown by the arrows inFIGS. 5-7. This linear movement allows the photo sensor 6 to be movedsuch that each column 6 a of the array is exposed to the same lightinput during the time span of the exposure. Each position the neutraldensity filter 8 is moved to varies from the previous by the width of apixel site or photo sensor site or multiple thereof. By varying theneutral density filter photo sensor position relative to the image 2 ofa smiley face rapidly during exposure, a set of images across a definedrange of exposure values can be captured and stored.

For example, in the photo sensor 6 shown in FIGS. 5-7, the neutraldensity filter 8 would be moved across the image. The image is capturedwhen the sensor is in three distinct positions, in which the point 12 inthe image 12 would be exposed to red and medium neutral density valuesas shown in FIG. 5, to red and lowest neutral density values as shown inFIG. 6, to red and highest neutral density values as shown in FIG. 7 andso on until each pixel site has been exposed to each of the varyingneutral filter densities, by linearly moving the neutral density filter.Each position of the photo sensor or the pixel site varies from theprevious by the width of a photo sensor site or natural multiple thereofso that the same point on the image 12 is captured by all color channelsand multiple exposure ranges.

A wide range of dynamic values can be obtained with additional neutraldensity arrays and other color patterns could be selected. The number ofrows or columns on the photo sensor could also vary based on theapplication. It is understood that the photo sensor array or arrangementof pixel sites shown in the figures is an unrealistically simple arrayand is used for explanation purposes only. It is understood that a Bayerfilter map arrangement may be used as well as other current photo sensorarrangements.

After the multiple color and exposure values are obtained, software orhardware to take the various input values and combine them into a singlehigh dynamic range image is used. The conversion to a high dynamic rangeimage utilizes the multiple exposure and color channel values obtainedby the photo sensor during the linear travel. No interpolation or“demosaicing” is required, nor do multiple exposure have to be alignedand combined to obtain high dynamic range values. The input values areall available from the input device and can be combined withoutintroducing additional estimation errors.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

Any suitable computer usable or computer readable medium may beutilized. The computer-usable or computer-readable medium may be, forexample but not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or propagation medium. More specific examples (a non-exhaustive list) ofthe computer-readable medium would include the following: an electricalconnection having one or more wires, a portable computer diskette, ahard disk, a random access memory (RAM), a read-only memory (ROM), anerasable programmable read-only memory (EPROM or Flash memory), anoptical fiber, a portable compact disc read-only memory (CD-ROM), anoptical storage device, a transmission media such as those supportingthe Internet or an intranet, or a magnetic storage device. Note that thecomputer-usable or computer-readable medium could even be paper oranother suitable medium upon which the program is printed, as theprogram can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited tothe Internet, wireline, optical fiber cable, RF, etc.

Computer program code for carrying out operations of the presentinvention may be written in an object oriented programming language suchas Java, Smalltalk, C++ or the like. However, the computer program codefor carrying out operations of the present invention may also be writtenin conventional procedural programming languages, such as the “C”programming language or similar programming languages. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

1. An imaging photo sensor comprising: a plurality of photo pixel sitesarranged in an array comprising a photo sensor and a neutral densityfilter filtering the photo sensor, each neutral density filter having adensity value, the density values of the neutral density filters beinggraduated over a range of densities; the neutral density filter beinglinearly movable adjacent to the photo sensor and moved relativethereto, wherein when the neutral density filter is moved across animage, each point in the image is exposed to at least one pixel sitehaving each of the graduated density values and each of the photo pixelsites of the array is exposed to a same light input during a time spanof exposure, such that the image is captured at a defined range ofexposure values; wherein multiple exposure pixel data captured atmultiple exposure values are combined into a single high dynamic rangeimage.
 2. The imaging photo sensor of claim 1, wherein the neutraldensity filter is mounted to a track.
 3. The imaging photo sensor ofclaim 1, the array is a Bayer map.
 4. The imaging photo sensor of claim1, wherein the neutral density filter is linearly moved into positionsthat vary from a previous position by a width of at least one photopixel site.
 5. A method of capturing high dynamic images comprising:aligning an imaging photo sensor comprising a plurality of photo pixelsites arranged in an array comprising a photo sensor and a neutraldensity filter filtering the photo sensor, each neutral density filterhaving a density value, the density values of the neutral densityfilters being graduated over a range of densities; moving the neutraldensity filter adjacent to the photo sensor and linearly relativethereto across the image, such that each point in the image is exposedto at least one pixel site having each of the graduated density valuesand each of the photo pixel sites of the array is exposed to a samelight input during a time span of exposure, capturing an image andcombining the captured multiple exposure pixel data taken at multipleexposures into a single high dynamic range.